ShAPO: Implicit Representations for Multi-object Shape, Appearance, and Pose Optimization

Our method studies the complex task of object-centric 3D understanding from a single RGB-D observation. As it is an ill-posed problem, existing methods suffer low performance for both shape and 6D pose size estimation in multi-object scenarios with occlusions. We present ShAPO, joint detection, textured reconstruction, object estimation. Key to ShAPO single-shot pipeline regress shape, appearance latent codes along masks each instance, which then further refined sparse-to-dense fashion. A novel disentangled database priors first learned embed objects their respective space. also propose novel, octree-based differentiable optimization step, allowing us improve simultaneously under space, analysis-by-synthesis implicit representation allows accurately identify reconstruct unseen without having access meshes. Through extensive experiments, we show that our method, trained on simulated indoor scenes, regresses real-world minimal fine-tuning. significantly out-performs all baselines NOCS dataset 8% absolute improvement mAP